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Abstract 

This study of Kentucky's 4th-grade NAEP reading scores was designed to: (1) describe progress 
from 2002-2007; (2) examine score gaps by gender, race, eligibility for free/reduced lunch, and 
75th/25th percentile 2002-2007, (3) explore relationships between teacher-quality and 2005/ 
2007 scores. The study, begun at 2007 NAEP data training, was completed using the NAEP 
DataTool. Results: (1) there were no statistically significant changes in the average scale score 
on the NAEP Kentucky fourth-grade reading test from 2002 through 2007, (2) gaps in 
performance by sub-groups on Kentucky's fourth-grade reading NAEP assessments were 
statistically significant and unchanged across the five-year period, and (3) there were no [ alpha 
.01] statistically significant differences in reading scores by teacher quality measures in either 
2005 or 2007. Since reading is a skill basic to both cognitive development and overall academic 
achievement, Kentucky appears to be at the point of crisis as the fourth-grade reading scores 
across years indicate no progress between the years 2002-2007. 
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Kentucky's Teacher Quality Measures and Fourth-Grade Reading Achievement: 

A Secondary Analysis of the 2002-2007 NAEP Data 

The purpose of this paper is threefold: (1) to describe the fourth-grade reading scores on 
the National Assessment of Educational Progress (NAEP) for Kentucky students in 2000-2007; 

(2) to examine the gaps and changes in gaps for selected subgroups of student scores in the 
datasets from 2002-2007, and (3) to explore the relationships between teacher quality measures 
and Kentucky students' fourth-grade reading achievement using the 2005 and 2007 NAEP data. 

Since the 1999 change from the Kentucky Instructional Results Information System 
(KIRIS) to the Commonwealth Accountability Testing System (CATS), the National Assessment 
of Educational Progress (NAEP) reading scores have been used as a validation measure of the 
statewide system. Thus, the NAEP Kentucky fourth-grade reading data for the years 2000-2007 
are being examined for indicators of improvement in learning (Koger, Thacker, Koger, & Deatz, 
2003). 

The No Child Left Behind Act (2001) included teacher quality as a major factor that is 
likely to affect student learning. Teacher quality was defined by NCLB basically as teacher 
credentials. These were (1) major/minor in elementary education, (2) highest academic degree, 

(3) type of teaching certificate, (4) years taught elementary school (NCLB, 2001). 

Researchers have recently reported that the quality of the teacher in the classroom was 
the most important schooling factor predicting student achievement (e.g., Darling-Hammond, 
2000; Darling-Hammond & Youngs, 2002; Hanhushek, Kain, & Rivkin, 1998).Wayne and 
Youngs (2003) found strong links between the NCLB-defined teacher quality variables and 
student achievement. Klecker (2007) found statistically significant relationships between teacher 
quality measures and eighth-grade mathematics achievement. 
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The National Assessment of Educational Progress (NAEP) has since 1969, been the only 
nationally representative and continuing assessment of what America's students know in various 
subject areas. The No Child Left Behind Act (2002) required participation of all school receiving 
Title I money from federal funds. This requirement has enlarged the data base. In 2005, teacher 
questionnaire data were added to include the NCLB-defined teacher quality variables (NCES, 
2008). 

Method 

Data Analysis 

The author of this study attended training to analyze the unique NAEP data in June 2007 
summer of 2007. During this visit, the data analyses of NAEP of 2005 data were performed. For 
the follow-up of the 2002-2007 and analysis of the teacher quality variables by 2007 NAEP 
scores, the researcher used —as advised by NAEP training staff— the NAEP Data Tool available 
on the NAEP website. [Note: IRB approval was obtained for this study.] 

Results 

History of Kentucky’ s NAEP Performance 
Education Trust (2002) reported: 

Between 1992 and 1998, Kentucky 4 th graders gained 5 points on the NAEP 4 th grade 
Reading assessment. The biggest state gain over that period was 10 points. From 1992- 
1998, the gap between White and African American students on the NAEP 4 th grade 
Reading increased by 7 points (p. 2). 
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Table 1 

Kentucky’s Fourth-Grade Reading Scores on NAEP 1998, 2002, 2003, 2005, 2007 



Year 


Average Scale Score 


SD 


1998 


218 


34 


2002 


219 


33 


2003 


219 


34 


2005 


220 


34 


2007 


222 


34 



The data in Table 1 indicate that there were no statistically significant 
(alpha =.01) changes in the average scale score on the NAEP Kentucky fourth-grade 
reading test from 1992 through 2007. Tables 2 through 5 present gaps and changes in gaps for 
selected subgroups (NCES, 2008). 

Tables 2 through 5 below present the data for the No Child Left Behind (NCLB) (2001) 
demographic categories. Table 2 presents the data for differences by gender. Table 3 presents the 
data for differences by race. Table 4 presents the data for differences by student free/reduced 
lunch, a measure of socioeconomic status and Table 5 presents the differences in scores between 



the 75th percentile and the 25th percentile. 
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Table 2. 

Kentucky NAEP Reading Grade Four Score Gaps by Gender 2002-2007 



Male Female 



Year 


Average Scale Score 


SE 


Average Scale Score 


SE 


Difference (SE) 


2002 


214.5 


(1.3) 


223.9 


(1.4) 


-2.9 (2.3) 


2003 


215.2 


(1.5) 


222.8 


(1.5) 


-7.6 (2.2) 


2005 


217.5 


(1.4) 


222.5 


(1.3) 


-4.9 (1.9) 


2007 


218.8 


(1.3) 


226.0 


(1.4) 


-7.2 (1.9) 



The changes in the gap year-by-year were not statistically significant, nor was the total change 
from 2002-2007. (The gap remained the same.) (NCES, 2008). 



Table 3. 

Kentucky NAEP Reading Grade Four Score Gaps by Race Across Years 





White 




Black 






Year 


Average Scale Score (SE) 


Average Scale Score (SE) 


Difference (SE) 


2002 


221.7 


(1.0) 


198.8 


(2.4) 


22.9 (2.6) 


2003 


221.4 


(1.2) 


201.6 


(3.3) 


19.8(3.5) 


2005 


221.9 


(1.1) 


203.3 


(2.3) 


18.6 (2.6) 


2007 


224.8 


(1.1) 


203.3 


(2.2) 


21.5 (2.5) 



There were no statistically significant differences in the changes in gaps between years or across 
years. (The gap remained the same.) (NCES, 2008). 
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Table 4. 

Kentucky NAEP Reading Grade Four Changes in Gaps by Free/Reduced Funch Status 





Not Eligible 


Eligible 




Year 


Average Scale Score (SE) 


Average Scale Score (SE) 


Difference (SE) 


2002 


229.0 (1.1) 


209.3 (1.5) 


19.7 (1.8) 


2003 


215.2 (1.5) 


209.3 (1.4) 


19.9 (2.2) 


2005 


228.0 (1.3) 


211.9 (1.2) 


16.1 (1.8) 


2007 


233.8 (1.3) 


211.7 (1.2) 


22.0(1.8) 



From 2002-2007 there was no statistically significant difference between or across the years. 
(The gap remained the same). (NCES, 2008). 



Table 5. 

Kentucky NAEP Reading Grade Four Gaps Between 75 th and 25 th Percentiles 





75 th Percentile 


25 th Percentile 






Year 


Average Scale Score (SE) 


Average Scale Score (SE) 


Difference (SE) 


2002 


242.3 (1.2) 


196.8 (1.8) 


45.5 


(2.2) 


2003 


243.0 (1.4) 


196.7 (1.4) 


46.3 


(1.9) 


2005 


243.6 (1.3) 


197.1 (1.3) 


46.5 


(1.7) 


2007 


245.9 (1.2) 


208.9 (1.3) 


45.0 


(1.7) 



From 2002-2007 the changes in the gap were not statistically significant difference between or 
across the years. (The gap remained the same.) (NCES, 2008). 
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From the data from years 2002 through 2007 presented in Tables 2 through 5, it is 
apparent that the changes in the gaps were not statistically significantly ( alpha =.01) different 
between or across the years. The gaps in performance by sub-groups on Kentucky's fourth-grade 
reading NAPE assessments were statistically significant (p. <.01) and remained unchanged 
across this five-year period. 

Tables 6 though 9 present the 2005 NAEP Kentucky fourth-grade reading scores by 
NCLB-defined teacher quality variables (1) major/minor elementary education, (2) highest 
academic degree, (3) type or teaching certificate, and (4) years taught elementary or secondary. 



Table 6. 

Major or Minor in Elementary Education by Fourth-Grade Reading Scores 2005 



Variable 


Average Scale Score 


SD 


Major in El. Ed. 


221 


35 


Minor/special emphasis 


218 


34 


No 


220 


33 



Tested with an alpha level of .01, there were no statistically significant differences between the 



levels of the variable. 
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Table 7. 

Highest Academic Degree by Fourth-Grade Reading Scores 2005 



Variable 


Average Scale Score 


SD 


Bachelor’s Degree 


111 


34 


Master’s Degree 


222 


35 


Education Specialist 


221 


32 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 



Table 8. 

Type of Teaching Certificate by Fourth-Grade Reading Scores 2005 



Variable 


Average Scale Score 


SD 


Regular/standard 


220 


34 


Probationary 


224 


33 


Provisional 


213 


31 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 
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Table 9. 

Years Taught Elementary or Secondary by Fourth-Grade Reading Scores 2005 



Variable 


Average Scale Score 


SD 


0-4 years 


216 


34 


5-9 years 


221 


33 


10-19 years 


222 


34 


20+ years 


222 


35 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 



Tables 10 through 13 below present the same comparisons using the 2007 NAEP scores. 
Tested with alpha at .01, there were no statistically significant differences in mean reading scores 
by teacher quality measures in either 2005 or 2007. 
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Table 10. 

Major or Minor in Elementary Education by Fourth-Grade Reading Scores 2007 



Variable 


Average Scale Score 


SD 


Major in El. Ed. 


233 


34 


Minor/special emphasis 


111 


40 


No 


224 


33 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 



Table 11. 

Highest Academic Degree by Fourth-Grade Reading Scores 2007 



Variable 


Average Scale Score 


SD 


Bachelor’s Degree 


219 


35 


Master’s Degree 


225 


34 


Education Specialist 


222 


33 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 
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Table 12. 

Type of Teaching Certificate by Fourth-Grade Reading Scores 2007 



Variable 


Average Scale Score 


SD 


Regular/standard 


223 


35 


Probationary 


220 


31 


Provisional 


227 


33 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 



Table 13. 

Years Taught Elementary or Secondary by Fourth-Grade Reading Scores 2007 



Variable 


Average Scale Score 


SD 


0-4 years 


223 


33 


5-9 years 


222 


26 


10-19 years 


223 


33 


20+ years 


222 


36 



Tested with an alpha level of .01, there were no statistically significant differences between the 
levels of the variable. 
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Conclusions 

The presentation of the Kentucky fourth-grade reading scores from the National 
Assessment of Educational Progress (NAEP) provided evidence that: 

• There was only a five point increase in the average scale score from 1992-1998 

• There was only a one point increase in the average scale score from 1998-2005 

• There were statistically significant gaps between scores by gender, race, SES, and 
percentile in 2002. 

• The 2003 and 2007 NAEP data show no changes in these gaps. 

• The teacher quality variables identified by NCLB were not related to any statistically 
significant differences in the average scale score. 

The scores on the National Assessment of Educational Progress (NAEP) have been used by 
Kentucky to validate the scores on the Commonwealth Accountability Testing System (CATS) 
since 1999. Based on the findings of this study, it is recommended that the reading teaching 
practices in Kentucky’s primary programs be re-examined. Further, the teaching of reading in 
Kentucky’s teacher education programs should be questioned. Since reading is a skill so basic to 
both cognitive development and overall academic achievement, Kentucky appears to be at the 
point of crisis as the fourth-grade reading scores across years indicate no progress in either 
teaching or learning between the years 2002-2007. Further, gaps in fourth-grade reading scores 
by gender, race, eligibility for free/reduced lunch, and 75 th /25 th percentile have not changed from 
2002 through 2007. These gaps are all statistically significant differences in scores and serve to 
further illuminate the stagnation of progress in 4 th grade reading in the state of Kentucky. 
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